Image sensor panel and method for capturing graphical information using same

ABSTRACT

The present disclosure provides an image sensor panel and a method for capturing graphical information using the image sensor panel. In one aspect, the image sensor panel includes a substrate and a sensor array on the substrate, the sensor array including a plurality of photosensitive pixels. The substrate includes a first region defined by the sensor array and a second region other than the first region. The second region is optically transparent and has an area greater than that of the first region.

RELATED APPLICATIONS

This application is continuation of U.S. application Ser. No.15/268,624, filed on Sep. 18, 2016, which claims the benefit of priorityto International Application No. PCT/US15/21199, filed Mar. 18, 2015,which claims the benefit of priority to U.S. Provisional Application No.61/955,223, filed Mar. 19, 2014, and U.S. Provisional Application No.62/025,772, filed Jul. 17, 2014, the entire contents of all both ofwhich are incorporated herein by reference for all purposes.

TECHNICAL FIELD

The present disclosure relates to an image sensor panel and a method forcapturing graphical information using the image sensor panel. Moreparticularly, the present disclosure relates to an image sensor panelincluding an array of photosensitive pixels, and a method for capturinggraphical information from a two-dimensional information bearingsubstrate (IBS) using the image sensor panel.

BACKGROUND

Flat panel displays have been used ubiquitously as a standard outputdevice for various electronic devices, such as, personal computers,laptop computers, smartphones, smart watches, televisions, handheldvideo game devices, a public information display, and the like.Recently, flat panel displays have been developed to include inputfunctionalities (e.g., touch screens that are sensitive to eitherpressure or capacitance changes in response to user touches orinteractions), such that the flat panel displays can be used as both aninput (pointer) device and an output device. A touch screen can interactwith the user and detect one or more of user's finger point contactsand/or drawings on the screen as input signals. However, a touch screencannot capture graphical information from the two-dimensional surface ofan information bearing substrate.

Many electronic devices, such as smartphones and laptop computers, havebeen developed to include a camera disposed before or behind their flatpanel displays pointing to a direction same or opposite to the lightemission direction of the displays. Although such a camera can be usedto capture images (still or moving pictures) as an input, the camerarequires focus and the captured images are often distorted or of lowquality due to, for example, a shaky hand.

Accordingly, there is a need to develop a new image sensor panel thatcan be easily integrated with a display panel (flat or curved) and caneasily capture graphical information from the two-dimensional surface ofan information bearing substrate without having to focus. There is alsoa need to develop a new method for capturing graphical, textual, orother information from the information bearing substrate using thecombination of an image sensor panel and a display panel.

SUMMARY

In one aspect, the present disclosure provides an image sensor panel,comprising a substrate; and a sensor array on the substrate, the sensorarray including a plurality of photosensitive pixels. The substratecomprises a first region defined by the sensor array and a second regionother than the first region. The second region is optically transparentand has an area greater than that of the first region. In oneembodiment, the substrate comprises one of glass, plastic, and acombination thereof.

In one embodiment, the image sensor panel further comprises a pluralityof column conductive lines and a plurality of row conductive linesintersecting the column conductive lines. The photosensitive pixels areformed proximate intersections of the column and row conductive lines.In one embodiment, the column and row conductive lines comprise anelectrically conductive material that is optically transparent.

In one embodiment, each of the photosensitive pixels has a stackedstructure comprising a control component and a photosensitive component.In one embodiment, the control component is formed on the substrate andthe photosensitive component is formed on the control component. In oneembodiment, the photosensitive component is formed on the substrate andthe control component is formed on the photosensitive component. In oneembodiment, the control component comprises three thin film transistors.

In one embodiment, the stacked structure further comprises a dielectriclayer between the control component and the photosensitive component,the dielectric layer having a via hole, and wherein the controlcomponent is electrically coupled to the photosensitive componentthrough the via hole. In one embodiment, the dielectric layer has athickness between 0.3 to 2.0 microns or at least 0.3 microns.

In one aspect, the present disclosure provides a two-dimensional imagescanner comprising a surface light source having a light-emittingsurface, and the image sensor panel as summarized above, the imagesensor panel being disposed on the light emitting surface of the surfacelight source. In one embodiment, the image scanner further comprises alight block component formed on the substrate within the first regionand between the light emitting surface of the surface light source andthe photosensitive pixels of the sensor array.

In one aspect, the present disclosure provides a bidirectional displaydevice, comprising a display module comprising a plurality of displaypixels separated by a black matrix, wherein the display pixels emitlight from a first surface of the display module; and the image sensorpanel as summarized above, the image sensor panel being attached to thefirst surface of display module. In one embodiment, the photosensitivepixels of the image sensor panel are aligned with the black matrix. Inone embodiment, the display pixels of the display module are alignedwith the second region of the substrate. In one embodiment, the imagesensor panel further comprises a light block component formed on thesubstrate of the image sensor panel within the first region and betweenthe display module and the photosensitive pixels of the sensor array.

In one aspect, the present disclosure provides a bidirectional displaydevice, comprising: a display module comprising a plurality of displaypixels configured to emit light from a first surface of the displaymodule; and an image sensor panel comprising a substrate and a pluralityof photosensitive pixels on the substrate, the substrate comprising afirst region defined by the photosensitive pixels and a second regionother than the first region, the second region being opticallytransparent and having an area greater than that of the first region;wherein the image sensor panel is attached to the first surface of thedisplay module.

In one embodiment, the display module comprises a liquid crystal display(LCD) module, and the display pixels are separated by a black matrix ofthe liquid crystal display module; and wherein the photosensitive pixelsof the image sensor panel are aligned to the black matrix of the displaymodule. In one embodiment, the image sensor panel further comprises alight block component formed on the substrate of the image sensor panelwithin the first region and between the display module and thephotosensitive pixels of the sensor array.

In one embodiment, the display module comprises an organic lightemitting diode (OLED) display and the display pixels of the displaymodule are aligned with the second region of the substrate. In oneembodiment, the image sensor panel further comprises a light blockcomponent formed on the substrate of the image sensor panel within thefirst region and between the display module and the photosensitivepixels of the sensor array. In one embodiment, the display pixels areconfigured to have a display resolution and the photosensitive pixelsare configured to have a sensor resolution, and wherein the sensorresolution is at least 1.5 times of the display resolution.

In one aspect, the present disclosure provides a method for capturinggraphical information from an information bearing substrate, the methodcomprising: contacting an information bearing substrate with a surfaceof an image sensor panel, the image sensor panel comprising an array ofphotosensitive pixels and an optically transparent region between thephotosensitive pixels; emitting probing light from a light source to theinformation bearing substrate through the optically transparent regionof the image sensor panel; detecting reflected light from theinformation bearing substrate using the photosensitive pixels to obtainraw image data; generating a digital image data file from the raw imagedata; and storing the digital image data file in a computer storagemedium.

In one embodiment, the method further comprises, after contacting theinformation bearing substrate with the surface of the image sensorpanel, determining a boundary of the information bearing substrate.

In one embodiment, emitting the probing light comprises emitting theprobing light from a portion of the light source that corresponds to asurface area defined by the boundary of the information bearingsubstrate.

In one embodiment, the method further comprises determining conformityof the information bearing substrate placed on the image sensor panel.

In one embodiment, the method further comprises: loading the digitalimage data file from the computer storage medium; and extracting textualinformation from the digital image data file by performing an opticalcharacter recognition process.

In one embodiment, the method further comprises: generating a compositedata file including image data and the extracted textual information;and storing the composite data file in the computer storage medium.

In one embodiment, the method further comprises: generating a text datafile including the extracted textual information; and storing the textdata file in the computer storage medium.

In one embodiment, emitting the probing light comprises emitting theprobing light having an intensity-time profile of a Gaussian type with ahalf-width of less than 1.0 second.

In one embodiment, emitting the probing light comprises sequentiallyemitting first color probing light, second color probing light, andthird color probing light to the information bearing substrate.

In one embodiment, detecting the reflected light comprises sequentiallydetecting first color reflected light, second color reflected light, andthird color reflected light from the information bearing substrate.

In one aspect, the present disclosure provides a computer programproduct stored in a computer memory, the computer program product whenexecuted by a processor causing the processor to perform the method forcapturing graphical information as summarized above.

In one aspect, the present disclosure provides an image sensor panel,comprising a transparent substrate having a first region and a secondregion, and an array of photosensitive pixels disposed on thetransparent substrate within the first region, the photosensitive pixelsbeing separated from each other by the second region. The second regionhas an area greater than that of the first region.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a sectional view of an input device including animage sensor panel, in accordance with an embodiment of the presentdisclosure.

FIG. 2 illustrates a plane view of an image sensor panel, in accordancewith an embodiment of the present disclosure.

FIG. 3 illustrates a sectional view of a sensor pixel of the imagesensor panel as illustrated in FIG. 2.

FIGS. 4A through 4D illustrate a layered structure of a sensor pixel,read by a single thin film transistor (TFT), in accordance with anembodiment of the present disclosure.

FIG. 5A illustrates a sectional view of an image sensor panel incombination with a liquid crystal display (LCD) module, in accordancewith embodiments of the present disclosure.

FIG. 5B illustrates a sectional view of an image sensor panel incombination with an organic light emitting diode (OLED) display module,in accordance with embodiments of the present disclosure

FIG. 6 illustrates a plane view of a sensor pixel array aligned with ablack matrix of a display module, in accordance with an embodiment ofthe present disclosure.

FIG. 7 illustrates a plane view of a sensor pixel array slightlymisaligned with a black matrix of a display module, in accordance withan embodiment of the present disclosure.

FIG. 8 illustrates a plane view of a sensor pixel array aligned with ablack matrix of a display module, in accordance with an embodiment ofthe present disclosure.

FIG. 9 illustrates a schematic circuit diagram of a sensor pixel, readby three thin film transistors (TFTs), in accordance with an embodimentof the present disclosure.

FIGS. 10A through 10D illustrate a layered structure of the sensor pixelas illustrated in FIG. 9.

FIG. 11 illustrates a perspective view of a laptop computer comprisingan image sensor panel in combination with a display module, whenscanning image of an information bearing substrate, in accordance withan embodiment of the present disclosure.

FIG. 12 illustrates a perspective view of a smartphone device comprisingan image sensor panel in combination with a display module, whenscanning image of an information bearing substrate, in accordance withan embodiment of the present disclosure.

FIG. 13 illustrates a perspective view of a smartphone device comprisingan image sensor panel in combination with a display module, whendetecting a user's finger drawing, in accordance with an embodiment ofthe present disclosure.

FIG. 14 illustrates a block diagram of an electronic device inaccordance with an embodiment of the present disclosure.

FIG. 15 illustrates a flow diagram for capturing graphical informationfrom an information bearing substrate in accordance with an embodimentof the present disclosure.

FIGS. 16A and 16B illustrate a bidirectional display device configuredas a 3D image sensor, in accordance with certain embodiments of thepresent disclosure.

DETAILED DESCRIPTION

The term “information bearing substrate” (or IBS) is used herein torefer to any tangible medium having a 2D surface that bears textual,graphical, or other information printed or otherwise attached thereto.In various embodiments, the information bearing substrate can be adocument, a photograph, a drawing, a business card, a credit card, asmartphone display screen, a surface of a merchandize package box, abook cover/page, a finger/palm/foot surface, and the like. Unlessotherwise provided, the term “image sensor panel” is used herein torefer to an image sensor panel device that includes a plurality ofphotosensitive pixels (or sensor pixels) formed on a glass/plasticsubstrate, as shown and described herein. Further, unless otherwiseprovided, the term “bidirectional display” is used herein to refer to apanel device that includes both a display panel and an image sensorpanel, as shown and described herein. In different cases, abidirectional display may be flat, flexible, or curved.

FIG. 1 illustrates a sectional view of an input device 1 including animage sensor panel 100, in accordance with an embodiment of the presentdisclosure. Referring to FIG. 1, input device 1 includes image sensorpanel (ISP) 100 and a backlight module 200, both being enclosed in ahousing 300 with a surface of ISP 100 being optically exposed to theexterior environment. Backlight module 200 comprises a light guide plate(having a square shape, a rectangular shape, or other suitable shapes)with a plurality of light reflectors 205 formed at a bottom surface ofthe light guide plate, and at least one light source 250, which can beone or more point light sources (e.g., LEDs) at one or more corners ofthe light guide plate or a linear light source (e.g., a cold cathodefluorescent lamp or CCFL) at a side of the light guide plate.

Backlight module 200 emits a uniform planar light source along a firstdirection 20 from a top surface of the light guide plate by guidinglight 210 from light source 250 and reflecting light 210 usingreflectors 205. In one embodiment, backlight module 200 includes asingle point light source 250, which emits light 250 of a white color.In another embodiment, backlight module 200 includes four point lightsources 250 disposed proximate the four corners of a square/rectangularlight guide plate, the light sources 250 respectfully emitting light of,for example, red, green, blue, and white (or infrared) colors.

In one embodiment, ISP 100 includes transparent areas such that theuniform planar light source generated from backlight module 200 canpenetrate therethrough and hit a document 10 placed on the top surfaceof the light guide plate. The planar light source is then reflected fromdocument 10 along a second direction 30. The reflected light opticallycarries information attached to document 10. Such information carried bythe reflected light can be detected by photosensitive pixels formed onthe ISP 100, as further detailed below.

FIG. 2 illustrates a plane view of an image sensor panel (ISP) 100, inaccordance with an embodiment of the present disclosure. ISP 100includes a transparent substrate 110, an array of sensor pixels 120, anda plurality of column conductive lines (columns) 130 and row conductivelines (rows) 140 electrically coupled with the sensor pixels 120. Sensorpixels 120 may be formed proximate intersections of columns 130 and rows140. In certain embodiments, sensor pixels 120 can be arranged on afirst region of substrate 110 to form a square lattice structure, arectangular lattice structure, a triangular lattice structure, ahexagonal lattice structure, and the like. Each of sensor pixels 120 canbe configured to have, for example, a circular shape, an oval shape, asquare shape, a rectangular shape having rounded corners, or any othersuitable shapes. In one embodiment, the first region of substrate 110 isrendered optically opaque due to the presence of sensor pixels 120. Inone embodiment, ISP 100 is devoid of light emitting elements andoptically transparent at non-sensor pixel regions (i.e., other than thefirst region).

In the embodiment of a square lattice structure, each sensor pixel mayhave a sensor pixel size S (e.g., a width or diameter, depending on thepixel shape, of about 10-40 um) and two neighboring sensor pixels may beseparated by a pixel separation distance P. Pixel separation distance Pmay be about 1.5 to 5 times of pixel size S. For example, pixel size Smay be 20 um, while pixel separation may be 30 um (P=1.5 S), 40 um (P=2S), or 50 um (P=2.5 S). The sensor pixels 120 are separated so as toleave transparent regions 150 (i.e., the non-sensor pixel regions) toallow at least a portion of the surface light source from backlightmodule 200 to penetrate therethrough.

FIG. 3 illustrates a sectional view of a sensor pixel 120 of the imagesensor panel 100 as illustrated in FIG. 2. Referring to FIG. 3, sensorpixel 120 may be formed on a TFT backplane 121 and include a bottomelectrode 122 on TFT backplane 121, an interlayer 123 on bottomelectrode 122, a photosensitive layer 124 on interlayer 123, a topelectrode 125 on photosensitive layer 124, and a protective layer 126 ontop electrode 125. Photosensitive layer 124 may comprise semiconductormaterials, e.g., amorphous silicon (a-Si), low temperature polysilicon(LTPS), metal oxide (ZnO, IGZO, etc.), and the like, which form a PINstructure. Alternatively, photosensitive layer 124 may comprise organicphoton sensitive materials. Interlayer 123 is optional and may comprisePEDOT:PSS. Protective layer 126 is optional and may comprise atransparent laminating material or alternatively a non-transparent(opaque) resin material so as to form a light block.

Referring to FIGS. 1 through 3, in one embodiment, a light source (e.g.,a surface light source from backlight module 200) can emit along a firstdirection 20 to an information bearing substrate (IBS) 10 bypassingsensor pixels 120 and through non-sensor pixel regions 150. The lightsource is then reflected from IBS 10, carrying information from IBS 10and entering into sensor pixels 120 along a second direction 30. In oneembodiment, darker markings on IBS 10 reflect less light (lowerintensity), while brighter markings on IBS 10 reflect more light (higherintensity). In response to the reflected light, photosensitive layer 124detects the information carried by the reflected light (e.g., intensityof the reflected light) and generates electrons, thereby forming a photocurrent that flows vertically and is detected or read through column 130and row 140. In this embodiment, top electrode 125 and/or protectivelayer 126 may comprise an optically non-transparent material, therebyacting as a light block, while TFT layer 121 and bottom electrode 122may comprise optically transparent materials.

It is appreciated that, in an alternative embodiment, first direction 20of light source and second direction 30 of reflected light may beopposite to those illustrated in FIG. 3. As such, in the alternativeembodiment, top electrode 125 and protective layer 126 may compriseoptically transparent materials, while bottom electrode 122 and/or TFTlayer 121 may comprise an optically non-transparent (or opaque)material, thereby acting as a light block.

FIGS. 4A through 4D illustrate a layered structure of a sensor pixel120, read by a single thin film transistor (TFT) 121, in accordance withan embodiment of the present disclosure. FIG. 4A illustrates the layeredstructure of a TFT backplane, before formation of a photosensitive layer124. FIG. 4B illustrates a section view of the layered structure takenfrom line B-B of FIG. 4A. FIG. 4C illustrates the layered structure,after formation of photosensitive layer 124 on the TFT backplane. FIG.4D illustrates a section view of the layered structure taken from lineD-D of FIG. 4D.

Referring to FIGS. 4A and 4B, sensor pixel 120 is formed on atransparent substrate 110 (made of, for example, glass or plastic). Agate electrode 121G is formed on substrate 110, and a first transparentdielectric layer 112 (made of, for example, SiO₂) is formed on gateelectrode 121G covering substrate 110. A channel layer 121C is formedand patterned on dielectric layer 112 and at a position that is alignedwith gate electrode 121G. Channel layer 121C comprises a semiconductormaterial of, e.g., amorphous silicon (s-Si), low temperature polysilicon(LTPS), metal oxide (ZnO, IGZO, etc.), and the like. Source/drainelectrodes 121SD and 121DS are formed on dielectric layer 112.Source/drain electrodes 121SD and 121DS are separated by channel layer121C, and electrically coupled to two opposing sides of channel layer121C. A second transparent dielectric layer 114 is formed on channellayer 121C and source/drain electrodes 121SD and 121DS. Gate electrode121G, channel layer 121C, source/drain electrodes 121SD and 121DSconstitute TFT 121.

A sensor pixel electrode 122 is formed on second transparent dielectriclayer 114 and is electrically coupled to source/drain electrode 121SDthrough a via hole 122V formed in dielectric layer 114. In thisembodiment, gate electrode 121G is electrically coupled to row line 140,while source/drain electrode 121DS is electrically coupled to columnline 130. In one embodiment, via hole 122V may be formed by selectivelyetching dielectric layer 114 to expose a portion of source/drainelectrode 121SD prior to the formation of sensor pixel electrode 122.Sensor pixel electrode 122 may be formed by depositing a metallicmaterial on dielectric layer 114, filling via hole 122V, andsubsequently patterned into, for example, a square shape having roundcorners, as shown in FIG. 4A. Prior to deposition of the metallicmaterial, a third transparent dielectric layer 116 may be formed andetched to define a shape of the pixel electrode 122. Upon deposition ofthe metallic material, dielectric layer 116 and sensor pixel electrode122 may be planarized for further processing.

Referring to FIGS. 4C and 4D, a photosensitive layer 124 is formed onpixel electrode 122, and a common electrode 125 is formed onphotosensitive layer 124. Common electrode 125 may be coupled to acommon ground (i.e., 0 Volt). In one embodiment, photosensitive layer124 may include a first semiconductor layer (having either n- or p-typeconductivity) deposited on pixel electrode 122, an intrinsicsemiconductor layer deposited on the first semiconductor layer, and asecond semiconductor layer (having a conductivity type opposite to thatof the first semiconductor layer) deposited on the intrinsicsemiconductor layer. The three layers form a photodiode having a PINstructure. As such, in response to impinging light, the PIN structurecan generate photocurrent that flows vertically. It is appreciated thatthe intrinsic semiconductor layer may not be necessary in certainapplications. In this embodiment, the photosensitive layer 124 is etchedand patterned in a photolithography process, so as to form an array ofsensor pixels 120 as shown in FIG. 2.

In other embodiments, the photosensitive layer 124 may be a dual-bandsensor, having two PIN structures (e.g., a vertical structure ofp-i-n-i-p or n-i-p-i-n, or a horizontal structure of p-i-n and p-i-n),one PIN structure being sensitive to, for example, visible light (havinga wavelength ranging between 400 nm and 700 nm), while the other PINstructure being sensitive to, for example, infrared light (having awavelength ranging between 700 nm and 1 mm). The two PIN structures canbe arranged vertically into a single stacked structure or horizontallyinto two stacked structures neighboring each other.

In one embodiment, the photosensitive layer 124 may comprise silicon,germanium, selenium, SiGe, GaAs, InGaAs, SiC, GaN, CuO, CuSe, CuTe, CdS,CdSe, CdTe, InSb, CuInGaS, CuInGaSe, CuInGaTe, TeGeHg, CuInSe, CuInS,CuInTe, HgCdTe, or combinations thereof in an amorphous, crystalline, orpolycrystalline form. For a sensor pixel array that is sensitive tovisible light, amorphous silicon (a-Si) p-i-n stack can be formeddirectly using a PECVD process. Additionally or alternatively, othersensing elements can be formed with a sensing function layer sandwichedbetween the top and bottom electrodes. For example, a thermal imagearray can be formed with a thermo-electric layer that is sensitive toinfrared light. In other embodiments, a scintillator film sensitive toX-ray can be formed in place of the p-i-n structure. See, for example,WO 2014/093244, published on Jun. 19, 2014, which is incorporated hereinby reference.

In an alternative embodiment, the photosensitive layer 124 may comprisesa mixture of a p-type polymeric organic semiconductor (e.g., poly(3-hexylthiophene) or poly (3-hexylthiophene-2, 5-diyl), also known asP3HT) and an n-type polymeric organic semiconductors (e.g.,[6,6]-phenyl-methyl-butanoate C6i, also known as PCBs). See, forexample, WO/2014/060693, published on Apr. 24, 2014, which isincorporated herein by reference.

FIGS. 5A and 5B respectively illustrate a sectional view of an imagesensor panel 100 in combination with a liquid crystal display (LCD)module 400 and an organic light emitting diode (OLED) display module500, in accordance with embodiments of the present disclosure. Referringto FIG. 5A, LCD module 400 includes a backlight module 410 configured toemit a planar light source 20 of, for example, white color. Backlightmodule 410 may additionally include a first polarizer so as to makeplanar light source 20 linearly polarized along a first direction. LCDmodule 400 further includes a TFT backplane 420 on backlight module 410,a liquid crystal material 430 on TFT backplane 420, and a color filterlayer 440 on liquid crystal material 430. The liquid crystal material430 is enclosed between TFT backplane 420 and color filter layer 440.TFT backplane 420 includes display pixel electrodes to control therotation orientation of liquid crystal material 430 so as to rotate thepolarization direction of planar light source 20 at different pixellocations. Color filter 440 includes an array of color pixels (e.g.,red, green, and blue) separated by a black matrix, each color pixelcorresponding to a display pixel electrode of TFT backplane 420, so asto generate colored output for LCD module 400. Color filter 400 mayadditionally include a second polarizer having a polarization directionperpendicular to that of the first polarizer of backlight module 410.

As shown in FIG. 5A, an image sensor panel (ISP) 100 is disposed oncolor filter 440 and is enclosed in a housing 300 together with LCDmodule 400. It is appreciated that, in certain embodiments, ISP 100 canbe formed and integrated with color filter 440. An upper surface of ISP100 is optically exposed to the exterior of housing 300 so as to allowlight source 20 to penetrate therethrough. As discussed above, ISP 100includes a plurality of photosensitive pixels separated from each otherto leave optically transparent regions. In one embodiment, thephotosensitive pixels of ISP 100 are aligned with the black matrix ofcolor filter 440 so as not to interfere with the display quality.

Although, in certain embodiments, ISP 100 of the present disclosure canbe used in touch control applications, display module 400 may stillinclude an optically transparent capacitive touch panel (not shown). Thecapacitive touch panel may be disposed on display module 400 and betweenISP 100 and color filter 440. Alternatively, the capacitive touch panelmay be disposed on ISP 100 and can be in contact with IBS 10 when placedthereon.

Referring to FIG. 5B, OLED display module 500 includes a TFT backplane510 having a plurality of light emitting pixels formed thereon and acover glass 520 on TFT backplane 510. As shown in FIG. 5B, an imagesensor panel (ISP) 100 is disposed on cover glass 520 and is enclosed ina housing 300 together with OLED display module 500. It is appreciatedthat, in certain embodiments, ISP 100 can be formed on and integratedwith cover glass 520. An upper surface of ISP 100 is optically exposedto the exterior of housing 300 so as to allow light source 20 topenetrate therethrough. As discussed above, ISP 100 includes a pluralityof photosensitive pixels separated from each other to leave transparentregions. In one embodiment, the transparent regions (i.e., the areabetween photosensitive pixels) of ISP 100 are aligned with lightemitting pixels of OLED display module, so as maintain display qualitysubstantially unaffected.

In some embodiments, display module 500 may additionally include anoptically transparent capacitive touch panel (not shown). The capacitivetouch panel may be disposed on display module 500 and between ISP 100and cover glass 520. Alternatively, the capacitive touch panel may bedisposed on ISP 100 and can be in contact with IBS 10 when placedthereon.

Referring to both FIGS. 5A and 5B, in operation, an information bearingsubstrate (IBS) 10, such as a document, can be placed on ISP 100 withthe information bearing surface contacting an upper surface of ISP 100,such that planar light source 20 can be reflected from IBS 10 to formreflected light 30. Sensor pixels of ISP 100 can then detect reflectedlight 30 from IBS 10, thereby capturing textual or graphical informationattached to the information bearing surface of IBS 10.

FIG. 6 illustrates a plane view of a sensor pixel array aligned with ablack matrix of a display module, in accordance with an embodiment ofthe present disclosure. FIG. 7 illustrates a plane view of a sensorpixel array slightly misaligned with a black matrix of a display module,in accordance with an embodiment of the present disclosure. FIG. 8illustrates a plane view of a sensor pixel array aligned with a blackmatrix of a display module, in accordance with an embodiment of thepresent disclosure.

Referring to FIGS. 6 through 8, different array configurations of sensorpixels 120 can be used in combination with color display pixels 440 tocapture optical images. In one embodiment, a color display pixel 440includes a red sub-pixel 442, a green sub-pixel 444, and a bluesub-pixel 446. Color display pixel 440 may have a pixel size D (e.g.,length or width) and a sensor pixel 120 may have a sensor pixel size S(e.g., diameter). As shown in FIG. 6, in one embodiment, sensor pixels120 are separated with each other by about D/3.

As shown in FIG. 7, in another embodiment, sensor pixels 120 areseparated with each other by about D/2. As shown in FIG. 8, in stillanother embodiment, sensor pixels 120 are separated with each other byabout 2D/3. As a result, the image sensor panel of the presentdisclosure can have a sensing resolution (monochrome) that is aboutdouble or triple of the display resolution. It is appreciated that colorfilters can be formed on sensor pixels 120 so as to capture colorfulimages using the image sensor panel of the present disclosure.

Referring to FIG. 6, sub-pixels 442, 444, and 446 are separated witheach other by a black matrix 448. In this embodiment, all sensor pixels120 are aligned with black matrix 448. Although, in certain cases,sensor pixel size S may be slightly greater than a width of black matrix448 and thus slightly overlap with display pixels 440, the displayquality is substantially unaffected, because at least 90% of thedisplay's light emitting area is not obstructed by sensor pixels 120.

Referring to FIG. 7, some of sensor pixels 120 are misaligned with blackmatrix 448. For example, as shown in FIG. 7, one of sensor pixels 120overlaps with green sub-pixel 444. Although the display's light emittingarea is somewhat obstructed by some sensor pixels 120, the displayquality (e.g., homogeneity) may still be substantially unaffected,because sensor pixel size S may be much smaller than display pixel sizeD. Further, the display quality can be adjusted and fine tuned bycalibrating the emission intension of individual display pixels 440after an image sensor panel is disposed and adhered onto a displaymodule.

Referring to FIG. 8, sub-pixels 442, 444, and 446 are separated witheach other by a black matrix 448 and all sensor pixels 120 are alignedwith black matrix 448. In this embodiment, sensor resolution is about1.5 times of the display resolution.

FIG. 9 illustrates a schematic circuit of a sensor pixel 120, read bythree thin film transistors (TFTs), in accordance with an embodiment ofthe present disclosure. In this embodiment, sensor pixel 120 includes afirst TFT 910, a second TFT 920, a third TFT 930, and a thin filmphotodiode 950. First TFT 910, second TFT 920, and third TFT 930 areformed on a glass substrate 110, while thin film photodiode 950 isstacked above first, second, and third TFTs 910, 920, and 930. As shownin FIG. 9, the gate of first TFT 910 is coupled to row 140. The drain offirst TFT 910 is coupled to column 130. The source of first TFT 910 iscoupled to the drain of second TFT 920. The source of second TFT 920 iscoupled to a voltage source VDD or 922. The gate of second TFT 920 iscoupled to a node 940. The gate of third TFT 930 is coupled to a resetsignal line 934. The source of third TFT 930 is coupled to a resetvoltage VRST or 932. The drain of third TFT 930 is coupled with node940. The cathode of photodiode 950 is coupled to node 940. The anode ofphotodiode 950 is coupled to a common ground 905.

Third TFT 930 acts as a switch to reset photodiode 950. When third TFT930 is turned on by an ON signal sent from reset signal line 934,photodiode 950 is connected to the reset voltage 932 (or power supply),thereby clearing charge accumulated on node 940. Second TFT 920 acts asa buffer (e.g., a source follower) and an amplifier, which allows pixelvoltage of photodiode 950 to be observed without removing theaccumulated charge. First TFT 910 acts as a switch to select one sensorpixel from an array of sensor pixels coupled with columns 130 and rows140, so as to be read the selected sensor pixel. When first TFT 910 isturned on in response to an ON signal from row 140, informationcorresponding to the pixel voltage of photodiode 950 can be measuredthrough column 130.

In one embodiment, voltage source VDD may be tied to the reset voltage932 (or power supply) of third TFT 930.

FIGS. 10A through 10D illustrate a layered structure of sensor pixel 120shown in FIG. 9. It is appreciated that, unless otherwise stated herein,the electrically conductive lines (e.g., signal lines, metallicinterconnects, etc.) discussed in the present disclosure have a linewidth of at least 5 um and a line separation of at least 3 um. Referringto FIGS. 10A through 10D together with FIG. 9, a gate electrode 911 offirst TFT 910, a gate electrode 921 of second TFT 920, and a gateelectrode 931 of third TFT 930 are formed on different locations ofsubstrate 110.

Referring to FIG. 10A, reset signal line 934 is formed on substrate 110,linearly traversing along a rear portion of sensor pixel 120, and iselectrically coupled to gate electrode 931 through interconnect 936. Rowconductive line 140 is formed on substrate 110, linearly traversing afront portion of sensor pixel 120, and is electrically coupled to gateelectrode 911 through a metallic interconnection 916. A metallicinterconnection 926 is formed on substrate 110 and is electricallycoupled to gate electrode 921.

Referring to FIGS. 10A and 10C, a first dielectric layer 960 having athickness of about 0.2 um to 0.4 um is formed over gate electrodes 911,921, and 931, reset signal line 934, row conductive line 140, andmetallic interconnections 916, 926, and 936. First dielectric layer 960includes a via hole 927 formed by selective etching, so as to expose aportion of metallic interconnection 926 at a distal end from gateelectrode 921. In one embodiment, first dielectric layer 960 comprises alow-k dielectric material and may be planarized for further processing.

Referring to FIGS. 10B and 10D, a channel layer 904 is formed on firstdielectric layer 960 above both gate electrodes 911 and 921, and achannel layer 934 is formed on first dielectric layer 960 above gateelectrode 931. In this embodiment, a single channel layer 904 is sharedbetween gate electrodes 911 and 921 for both first and second TFTs 910and 920. In other embodiments, instead of a single channel layer 904,two separate channel layers may be formed on first dielectric layer 960respectively above and aligned with gate electrodes 911 and 921.

Referring to FIGS. 10B, 10C, and 10D, column conductive line 130 isformed on first dielectric layer 960, linearly traversing a right handportion of sensor pixel 120, and is electrically coupled to a right sideof channel layer 904 (or first TFT 910) through source/drain electrode913. Voltage source line 922 is formed on first dielectric layer 960,linearly traversing a central portion of sensor pixel 120, and iselectrically coupled to a left side of channel layer 904 (or second TFT920) through source/drain electrode 925. Reset voltage line 932 isformed on first dielectric layer 960, linearly traversing a left handportion of sensor pixel 120, and is electrically coupled to a left sideof channel layer 934 (or third TFT 930) through source/drain electrode935. A metallic interconnection (or source/drain electrode) 933 isformed on first dielectric layer 960 filling via hole 927, and iselectrically coupled between a right side of channel layer 934 and gateelectrode 921 through metallic interconnection 926.

In one embodiment, when channel layer 904 includes a single layer whichcomprises a semiconductor material of relatively low electrical mobility(e.g., a-Si having an electron mobility of about 0.5˜1.0 cm²/(V·s) andIGZO having an electron mobility of about 1.0˜20.0 cm²/V·s), a metallicinterconnection 905 is formed on channel layer 904 and between gateelectrodes 911 and 921, so as to constitute source/drain electrodes 923and 915, and to form two distinctive channel regions respectfully forfirst and second TFTs 910 and 920. In another embodiment, when channellayer 904 includes a single layer which comprises a semiconductormaterial of relatively high electrical mobility (e.g., LTPS having anelectric mobility of at least 100.0 cm²/V·s), metallic interconnection905 is not necessary for first and second TFTs 910 and 920. In analternative embodiment, when channel layer 904 includes two separate andmutually insulating channel layers, a metallic interconnection 905 isformed on first dielectric layer 960 and over sides of the two separatechannel layers, so as to constitute source/drain electrodes 923 and 915.

Referring to FIGS. 10B and 10D, a second dielectric layer 970 is formedon source/drain electrodes 913, 915, 923, 925, 933, and 935, and overchannel layers 904 and 934. In one embodiment, second dielectric layer970 includes a via hole 942 formed by selective etching of seconddielectric layer 970 and may be planarized for further processing. Asshown in FIG. 10D, a sensor pixel electrode 122 is formed on seconddielectric layer 970 filling via hole 942, and is electrically coupledto third TFT 930 through source/drain electrode 933. In one embodiment,sensor pixel electrode 122 may have a surface area substantiallyequivalent to that of a collection of first, second, and third TFTs 910,920, and 930, so as to form a vertically stacked structure. In oneembodiment, second dielectric layer 970 has a thickness ranging between0.3 um and 2.0 um or at least 0.3 um, so as to prevent or at leastreduce undesirable interferences to the underlying TFTs 910, 920, and930 due to, for example, a parasitic capacitance effect.

FIG. 11 illustrates a perspective view of a laptop computer 1100comprising a bidirectional display 1110 (including a display panel andan image sensor panel disposed on the display panel) in accordance withan embodiment of the present disclosure. Bidirectional display 1110 canbe used to capture graphical information on an information bearingsubstrate (IBS) 1120. As shown in FIG. 11, a user can use one hand tohold IBS 1120 with the information bearing surface (i.e., the surfacehaving, for example, texts or graphics attached thereto) facing a screensurface of bidirectional display 1110. The user can use another hand totrigger a scan event by pressing a button 1130 (e.g., a key on thekeyboard) of laptop computer 1100. In an alternative embodiment, thescan event may be automatically triggered once IBS 1120 contacts andproperly conforms to the screen surface of bidirectional display 1110.

The information attached to IBS 1120 can be optically captured bybidirectional display 1110 in response to the triggered scan event. Inone embodiment, laptop computer 1100 may additional include a mechanicalswitch at a side of the keyboard portion of laptop computer 1100 to turnon/off of the electrical power for the image sensor panel ofbidirectional display 1110, so as to reduce power consumption andenhance user privacy. It is appreciated that the mechanical switch maybe implemented as a soft/virtual button and/or a function key or acombination of function keys on the keyboard.

FIG. 12 illustrates a perspective view of a smartphone device 1200comprising a bidirectional display 1210 (including a display panel andan image sensor panel disposed on the display panel) in accordance withan embodiment of the present disclosure. Bidirectional display 1210 canbe used to optically capture an image copy of an information bearingsubstrate 1220. Referring to FIG. 12, smartphone device 1200 includesbidirectional display 1210, a mini camera 1202 proximate a top side ofsmartphone device 1200, a start button 1240 proximate a bottom side ofsmartphone device 1200, and a power/lock button 1260 at a side ofsmartphone device 1200. Smartphone device 1200 may optionally include aprotective cover flap 1205 and optionally include an earphone 1250having a control button 1230.

As shown in FIG. 12, a user can hold the IBS 1220 with theinformation-bearing surface facing a screen surface of bidirectionaldisplay 1210. In one embodiment, protective cover flap 1205 may includean IBS holder 1206, which can be sized to accommodate, for example, astandard business card or credit card, such that IBS 1220 can be held byprotective cover flap 1205 and scanned when protective cover flap 1205is flipped and closed onto bidirectional display 1210. Depending ondesign choices, the user can trigger a scan event by pressing startbutton 1240, pressing control button 1230 of earphone 1250, pressingpower/lock button 1260, or a combination of the above. In anotherembodiment, a scan event may be triggered immediately after IBS 1220contacts and properly conforms to the screen surface of bidirectionaldisplay 1210.

In yet another embodiment, smartphone device 1200 may include one ormore magnetic switches (not shown) disposed at one or various positionsunderneath bidirectional display 1210. In this embodiment, a user maytrigger a scan event by closing cover flap 1205 onto bidirectionaldisplay 1210, and approaching and aligning one or more magnets to themagnetic switches. The magnets can be embedded in cover flap 1205 orapplied externally after cover flap 1205 is covered on bidirectionaldisplay 1210.

As shown in FIG. 12, IBS 1220 may have a substantially rectangularshape, but when placed on the display screen slightly rotated relativeto the rectangular shape of bidirectional display 1210. Accordingly, inone embodiment, smartphone device 1200 can first detect the boundary ofIBS 1220. Upon determination of the boundary of IBS 1220, which may beslightly bigger than the actual area of IBS 1220, the display panel ofbidirectional display 1210 can emit a probing light only within the IBSboundary. In response to triggering of the scan event, informationattached to IBS 820 can be carried by reflected light, captured by theimage sensor panel of bidirectional display 1210, and converted into adigital image.

FIG. 13 illustrates a perspective view of a smartphone device 1300comprising a bidirectional display 1310 (including a display panel andan image sensor panel disposed on the display panel). Bidirectionaldisplay 1310 can be used to detect a user's finger drawing 1320, inaccordance with an embodiment of the present disclosure. Referring toFIG. 13, similar to smartphone device 1200 of FIG. 12, smartphone device1300 includes bidirectional display 1310 and a start button 1340proximate a bottom side of smartphone device 1320. Instead of emitting aprobing light pulse, in a drawing detection mode, the image sensor panelof bidirectional display 1310 can passively detect the user'scontacts/drawings using the image sensor panel. Alternatively, thedisplay panel of bidirectional display 1310 can actively emit a constantintensity probing light during a time period that bidirectional display1310 is switched to the drawing detection mode. In one embodiment, whenan object (such as a finger tip or stylus tip) is placed on or proximatebidirectional display 1310, it can sequentially sense the locationsand/or directions of the object forming drawing 1320 on bidirectionaldisplay 1310, thereby detecting a shape or pattern of drawing 1320. Incertain embodiments, the image sensor panel of bidirectional display1310 can be used in place of the existing capacitive touch sensitivepanel.

FIG. 14 illustrates a block diagram of an electronic device 1400 inaccordance with an embodiment of the present disclosure. As shown inFIG. 14, electronic device 1400 includes a bidirectional display 1410, amemory 1420, a processor 1430, an input device 1440, and a storagedevice 1450, which are electrically coupled with each other through asystem bus 1460. Bidirectional display 1400 includes a display panel 400or 500, and an image sensor panel 100 disposed on display panel 400 or500, examples of which are illustrated in FIGS. 5A and 5B. In addition,bidirectional display 1410 includes a source control circuit 1404, agate control circuit 1406, a column decode circuit 1402, and a rowdecode circuit 1408. Source control circuit 1404 and gate controlcircuit 1406 are electrically coupled to display panel 400 or 500through source lines and gate lines, so as to control emission of thelight emitting elements of display panel 400 or 500. Column decodecircuit 1402 and row decode circuit 1408 are electrically coupled tosensor pixels of image sensor panel 100 through column decode lines androw decode lines, so as to control light detection of the sensor pixelsof image sensor panel 100.

In one embodiment, source control circuit 1404, gate control circuit1406, column decode circuit 1402, and row decode circuit 1408 arerespectively implemented in different integrated circuit (IC) chips. Itis appreciated, however, that some or all of source control circuit1404, gate control circuit 1406, column decode circuit 1402, and rowdecode circuit 1408 may be integrated in a single IC chip. Further,source control circuit 1404 and gate control circuit 1406 may beimplemented as one IC chip, while column decode circuit 1402 and rowdecode circuit 1408 may be implemented as another IC chip.

In order to capture images using bidirectional display 1410, a computerprogram product may be stored in storage device 1450 (e.g., hard drive,non-volatile solid state memory drive, and the like) and loaded tomemory 1420 (e.g., random access memory (RAM) or other volatile memory)of electronic device 1400 for execution by processor 1430. In oneembodiment, all or part of the computer program product may be includedas add-on modules in an operating system (e.g., Linux, Android, iOS,etc.) for the electronic device 1400. In another embodiment, thecomputer program product may be a standalone computer softwareapplication (e.g., a mobile app or a software application package)executable on electronic device 1400 using resources of the operatingsystem. In one embodiment, the computer program product includes aprobing module 1421, a light detection module 1422, and an imageassembly module 1425. In one embodiment, the computer program productadditionally includes one or more of a boundary determination module1423, a conformity determination module 1424, and an optical characterrecognition (OCR) module 1426. In one embodiment, the computer programproduct can further include an interpreter module 1428, such as ascripting language software program configured to interpret lines ofcodes written in a specific scripting language and to instruct hardwareof electronic device 1400 to perform the functions as provided in thelines of codes.

Probing module 1421 comprises computer instructions to control gatecontrol circuit 1406 and source control circuit 1404, such that probingsignals can be transmitted to the light emitting elements of displaypanel 400 or 500, in response to a scan event, thereby emitting apredetermined probing light.

Light detection module 1422 comprises computer instructions to controlcolumn decode circuit 1402 and row decode circuit 1408, such that light(e.g., reflected light from the IBS) entering the sensor pixels of imagesensor panel 100 can be detected in response to a scan event. Thedetected raw image data may be transmitted to image assembly module 1425for further processing.

Image assembly module 1425 comprises computer instructions to analyzethe raw image data detected by light detection module 1422, and assembleor generate a digital image data file from the raw image data. In oneembodiment, the digital image data file can be a pixel-based image datafile of a lossy/lossless compressed format (e.g., JPG, PNG, TIFF, etc.)or a pixel-based image data file of an uncompressed format (e.g., BMP).In one embodiment, image assembly module 1425 saves the generateddigital image data file into storage device 1450. The digital image datafile is then ready for user access or further processing.

In one embodiment, the generated digital image data file is transmittedto or loaded by OCR module 1426 for further processing. OCR module 1426analyzes the digital image data file and extracts textual informationfrom the image data file. The extracted textual information can be savedinto storage device 1450 as a text data file (e.g., TXT, DOC, etc.).Alternatively, the extracted textual information can be saved togetherwith non-textual image data as a combined image/text data file (e.g.,PDF).

Boundary determination module 1423 comprises computer instructions todetermine the boundary of an IBS placed on an image sensor panel 100 ofthe present disclosure. In one embodiment, upon triggering of a scanevent, boundary determination module 1423 transmits an instruction setto detection module 1422 to passively monitor an IBS. Upon placing anIBS over a screen surface of image sensor panel 100, the boundarydetermination module 1422 captures a temporary image for the entirescreen of image sensor panel 100. Because detection module 1422passively monitors an IBS (i.e., without emitting a probing light), arelatively darker region in the temporary image may correspond to theIBS region, while a relatively brighter region in the temporary imagemay correspond to the non-IBS region. It is appreciated that detectionmodule 1422 can monitor an IBS with a white probing light beingconstantly emitted. In such cases, a relatively brighter region in thetemporary image may correspond to the IBS region, while a relativelydarker region in the temporary image may correspond to the non-IBSregion.

In one embodiment, boundary determination module 1423 analyzes thetemporary image and determines a boundary of the IBS in accordance witha brightness change in the temporary image. In one embodiment, prior toanalyzing the temporary image, boundary determination module 1423 canadjust the temporary image by, for example, increasing contrast of thetemporary image. In one embodiment, boundary determination module 1423generates a boundary data file (comprising data points of the boundarylocations) and save the boundary data file in storage device 1450. Inone embodiment, the IBS may be a user's foot, and by placing the user'sfoot on image sensor panel 100 of the present disclosure, boundarydetermination module 1423 can acquire boundary data of the user's foot.The boundary data can then be further analyzed and converted to a shoesize of the user, thereby determining inventory availability of footwearfor the user.

Conformity determination module 1424 comprises computer instructions todetermine whether an IBS has been properly placed on a screen surface ofimage sensor panel 100. In one embodiment, conformity determinationmodule 1240 monitors the sensor pixels of image sensor panel 100 andwaits for an IBS. Once the sensor pixels detect that an IBS is placed onimage sensor panel 100, a temporary image may be captured andtransmitted to conformity determination module 1424. Conformitydetermination module 1424 then analyzes the temporary image anddetermines whether the temporary image is substantially focused. Becauseno converging or diverging lens is included in bidirectional displaypanel 1410 of the present disclosure, the temporary image can be focusedor not blurred only when the IBS contacts or is disposed very close toan upper surface of image sensor panel 100. If the temporary image issubstantially focused or not blurred, conformity determination module1424 then determines that the IBS is properly placed on image sensorpanel 100 and ready for the scan event.

FIG. 15 illustrates a flow diagram for capturing graphical informationfrom an information bearing substrate in accordance with an embodimentof the present disclosure. The method of the present disclosure cancapture graphical information on a monochromic (black/white or greyscale) information bearing substrate or a colored information bearingsubstrate.

In one embodiment, in Step 1501, an information bearing substrate (IBS)is placed in contact or in close proximity to a screen surface of abidirectional display with an information-bearing surface facing thebidirectional display. In Step 1503, in response to a triggered scanevent, a probing light of a single color is emitted from thebidirectional display panel to the IBS. In Step 1505, reflected lightfrom the IBS is detected by an array of light detecting elements of thebidirectional display panel as raw image data. In Step 1507, a digitalimage data file of a predetermined format is constructed from the rawimage data. In Step 1509, the digital image data file is stored in acomputer memory for further processing.

In an alternative embodiment, Steps 1503 and 1505 can be repeatedseveral times for capturing information on a colored IBS. For example,by sequentially (i) emitting red probing light and detecting reflectedred light, (ii) emitting green probing light and detecting reflectedgreen light, and (iii) emitting blue probing light and detectingreflected blue light, the bidirectional display of the presentdisclosure can generate colored raw image data for constructing acolored digital image data file.

FIGS. 16A and 16B illustrate a bidirectional display 1600 configured asa 3D image sensor, in accordance with certain embodiments of the presentdisclosure. In one embodiment, image sensor panel 1610 of bidirectionaldisplay 1600 may be divided into sub-regions using a software programproduct to capture images of an object from different angles. In oneembodiment, bidirectional display 1600 may be a curved display having aconcave active surface, such that different sub-regions of image sensorpanel 1610 can take images of an object at different angles. Such imagestaken from different angles of the same object may be used toreconstruct a 3D image of the object.

In one embodiment, bidirectional display 1600 may be a flat paneldisplay having a flat display surface. For those flat panel devices, theimage sensor panel of display device 1600 may additional include a microlens array having a plurality of microlenses, each microlens beingformed on an image sensor pixel, and the microlenses together forming aneffective optical lens (e.g., a Fresnel lens) for certain desiredoptical functionality (e.g., focus, field of view, etc.) for the entireimage sensor panel or sub-regions thereof.

Referring to FIG. 16A, in one embodiment, image sensor panel 1610 can bedivided into two sub-regions, i.e., a left input region 1610L(effectively a first camera) and a right input region 1610R (effectivelya second camera). Left and right input regions 1610L and 1610R may be arectangular area having equal widths and lengths. In one embodiment,left and right input regions 1610L and 1610R may have an area that is afraction of the total display area. In one embodiment, left and rightinput regions 1610L and 1610R may have the same aspect ratio as that ofbidirectional display 1600, such that images taken by regions 1610L and1610R can be displayed directly on display device 1600 withouthorizontal or vertical scaling/distortion.

Referring to FIG. 16B, in one embodiment, image sensor panel 1610 can bedivided into four sub-regions, i.e., an upper-left input region 1610UL(effectively, a first camera), a lower-left input region 1610LL(effectively, a second camera), an upper-right input region 1610UR(effectively, a third camera), and a lower-right input region 1610LR(effectively, a fourth camera). Similar to input regions 1610L and 1610Rshown in FIG. 16A, input regions 1610UL, 1610LL, 1610UR, and 1610LR maybe a rectangular area having equal widths and lengths (i.e., equalarea). In one embodiment, input regions 1610UL, 1610LL, 1610UR, 1610LRmay have an area that is a fraction of the total display area ofbidirectional display 1600. In one embodiment, input regions 1610UL,1610LL, 1610UR, and 1610LR may have the same aspect ratio as that ofbidirectional display 1600, such that images taken by input regions1610UL, 1610LL, 1610UR, and 1610LR can be displayed directly on displaydevice 1600 without horizontal or vertical scaling/distortion. Althoughtwo kinds of sub-region division are shown and described, it isappreciated that other types of sub-region division are possible withoutdeparting from the spirit and scope of the present disclosure.

The bidirectional display of the present disclosure can be used invarious contexts. Below are several of such exemplary usage cases. Itshould be appreciated that the bidirectional display of the presentdisclosure can be used in various other contexts without departing fromthe spirit and scope of the present disclosure.

Example One—Credit Card

In this example, a consumer's credit card is an IBS. Before completionof an online purchase, for example, a consumer is usually prompted toenter credit card information through an electronic commerce website ora mobile app. The required credit card information includes one or moreof the credit card number (e.g., 16 digits), the expiration month/year,and the account holder's first and last names appearing on one side ofthe credit card, and the security code (3 or 4 digits) appearing on theother side of the credit card.

With the bidirectional display of the present disclosure, an electroniccommerce website or mobile app can employ a graphic user interface onthe display panel to capture credit card information directly from thefront and back surfaces of the consumer's credit card. As such, theconsumer does not need to manually enter the credit card information atthe conclusion of the online purchase. Moreover, the bidirectionaldisplay of the present disclosure can eliminate the need of a magneticstripe reader for credit card transactions.

In one embodiment, the graphic user interface on a bidirectional displayincludes a first rectangular area, which may have a physical areacomparable to or slightly greater than the surface dimension of astandard credit card. The graphic user interface can then instruct theconsumer to place his/her credit card on the bidirectional display andwithin the boundary of the first rectangular area, with the front orback surfaces of the credit card facing the bidirectional display. Ascan event can then be triggered such that the electronic commercewebsite or mobile app can capture an image of the front or back surfacesof the credit card.

In one embodiment, the captured image may be analyzed using an OCRmodule to acquire textual credit card information from the capturedimage. The textual credit card information may include credit cardnumber, account holder name, expiration month/year, security code, etc.Thereafter, monetary transaction for the online purchase can becompleted using the acquired credit card information. In one embodiment,the captured information can additional include an image of theconsumer's hand-written signature or an image of the laser sticker on aside of the credit card, thereby enhancing the transaction security inelectronic commerce.

Example Two—Footwear Size Measurement

In this example, a consumer's foot is an IBS. Online purchase of attiresand fashion accessories has become common practice for consumers withInternet access. When ordering, for example, footwear (especiallychildren shoes) in an online purchase, it is often difficult to selectthe correct shoe size for the desired footwear style. Oftentimes, theshoe size numbering systems are inconsistent among different shoemakers/vendors, especially when they are based in different countries.That is, the same shoe size number of two shoe makers/vendors maycorrespond to two totally different actual physical sizes. Accordingly,a consumer often orders footwear of a wrong size and have to spend extratime and effort to return/exchange for a correct size.

With the bidirectional display of the present disclosure, real size ofan object can be accurately measured. For example, an electroniccommerce website or mobile app can employ a graphic user interface onthe bidirectional display and acquire foot size information through thegraphic user interface. In one embodiment, the graphic user interfaceprompts the consumer to place a bare foot (such as a baby foot) on abidirectional display. In an alternative embodiment, the consumer canfirst draw a foot print on a piece of paper by tracing an actual footthereon, and then place the foot print paper on the bidirectionaldisplay for shoe size determination. A scan event can then be triggeredsuch that foot print information can be captured directly from thebidirectional display.

Thereafter, the electronic commerce website or mobile app can analyzethe foot print information and determine a foot size, therebydetermining a correct shoe size number for the desired footwear.Although the foot size determination may not be completely accurate,with the assistance of the bidirectional display of the presentdisclosure, the probability of ordering wrong sized footwear can belargely reduced.

Example Three—Optical Touch

When a user's finger tip contacts or touches a screen surface of abidirectional display, the soft and flexible finger tip become flattenedand shows a circular/elliptical shape (or a touch shape) on the displayscreen. The photosensitive pixel array can “see” the touch shape andtrigger a TOUCH_DOWN event in the electronic device of the presentdisclosure. When the finger leaves the screen surface of thebidirectional display, it triggers a TOUCH_UP event in the electronicdevice of the present disclosure. For example, one sequence ofTOUCH_DOWN and TOUCH_UP events within a pre-determined time periodconstitutes a SINGLE_CLICK event. For example, two sequential TOUCH_DOWNand TOUCH_UP events within a pre-determined time period constitutes aDOUBLE_CLICK event. When the finger touches the screen surface of thebidirectional display and traverses on the screen surface, it triggers aDRAWING event, and the electronic device can then detect the drawingshape as an input.

Example Four—Fingerprint and Other Biometrics

A smartphone device having a bidirectional display (including a displaymodule and an image sensor panel) can be used to detect a user'sfingerprint and/or other biometrics. A portion (e.g., asquare/rectangular area) of the bidirectional display can be used tooptically detect a fingerprint, while another portion/area of thebidirectional display can show the detected result in real time. Thedetected fingerprint can be compared with an existing finger print datastored in an electronic device to control access of the electronicdevice.

Example Five—Pulse Oximeter

A smartphone device having a bidirectional display (including a displaypanel and an image sensor panel) can be used to detect pulse rate orheard beat of a human being. For example, a user can place and press afinger (e.g., index finger or thumb) on the bidirectional display of asmartphone device. Then, the display panel can emit a light source of,for example, red color, to the finger and the image sensor panel canmeasure the light diffused in and reflected from the finger. Thesmartphone device can then analyze the temporal behavior of thereflected light detected by the image sensor panel to determine theheartbeat or pulse rate of the user.

As discussed above, a smartphone device having a bidirectional displayof the present disclosure can also be used to capture a fingerprint. Ithas been reported that existing fingerprint scanners can be tamperedeasily by using an image copy of the user's finger. As such,simultaneous measurement of both fingerprint and pulse rate can ensurethat the fingerprint is captured from an actual and living person.Accordingly, the bidirectional display of the preset disclosure canachieve enhanced security to prevent unauthorized access of thesmartphone device or other transactions.

Example Six—Thermometer

In one embodiment, a display screen of a smartphone device can includean image sensor panel of the present disclosure. The image sensor panelcan include dual-band sensor pixels that are sensitive to both infraredand visible light. Because only a small number of infrared sensitivepixels are required to take temperature measurement, it is appreciatedthat not all of the sensor pixels on the image sensor panel need to besensitive to infrared light. In this embodiment, a user can simply placea display screen of a smartphone device in contact with the user's bodysurface (e.g., forehead) to measure black body radiation of the user'sbody, thereby determining the body temperature of the user.

Example Seven—Data Entry

In this example, an IBS can be a document, a sheet of paper, a businesscard, and the like, on which encoded markings, such as a QR-code, a barcode (1D or 2D), and the like, are printed or otherwise attachedthereon. The encoded markings can be decoded using a pre-determinedalgorithm. It is appreciated that a smartphone screen can also serve asan IBS, and the encoded markings can be displayed on the smartphonescreen.

Various types of information can be included in the encoded markings.For example, the encoded markings can include address information of adesired destination (e.g., longitude, latitude, and/or sea level, orstreet address), computer instructions in textual form (e.g., computercodes written in a scripting language), media data in binary form (e.g.,video data, audio data, image data, etc.), and the like. Any dataincluded in the encoded markings on an IBS can be read by placing theIBS on an image sensor panel of the present disclosure. It isappreciated that, although focus is required, a rear or front camera ofa smartphone device can also be used to read the IBS described herein.

In one embodiment, a vehicle navigation system can include an imagesensor panel on its display screen and the encoded markings of an IBScan include destination information (e.g., street address, or latitudeand longitude). In operation, a user can place the IBS proximate to orin contact with the screen surface of the vehicle navigation system. Thevehicle navigation system then reads the destination information fromthe IBS and, in response, automatically finds and shows the destinationon its display screen within only a few seconds. The user can click aconfirm/start button on the navigation system, thereby beginningnavigation to the destination.

In an alternative embodiment, a smartphone device can include an imagesensor panel on its display screen and the encoded markings of an IBScan include computer instructions written in a scripting language. It isappreciated that the IBS having encoded markings can also be read usingthe rear or front camera of a smartphone device.

For example, an IBS can include encoded markings of computerinstructions to be used in an emergency situation, such as, a caraccident. Such computer instructions can be, for example:

<Begin> Username = “John Doe”; InsuranceNo = “+1-202-321-9876”; PolicyNo= “001122334455”; Police = “911”; // Pre-defined variables Loc =GetLocation ( ); // Get current location information and output resultto variable Loc = [X, Y], wherein X=latitude, Y=longitude Time =GetCurrentTime ( ); // Get current date and time and output result tovariable Time Contact = GetPhoneNumber ( ); // Get phone number of thepresent device and output to variable Contact Msg1 = Call (Police,“$Username$ had a car accident at location $Loc$ on $Time$ and his/hercontact number is $Contact$, please help!”); // Call police officer andprovide information about the user's name, accident location and time,and the user's contact number. The output (e.g., success or fail) issaved in variable Msg1. Msg2 = Text (Insurance, “$Username$, whosepolicy number is $PolicyNo$ had a car accident at location $Loc$ on$Time$”); // Send text message to insurance company and provideinformation on the user's name, insurance policy number, and accidentlocation and time. The output (e.g., success or fail) is saved invariable Msg2. <End>

When the IBS is scanned and read by an image sensor panel on asmartphone screen, for example, the above computer instructions triggeran interpreter module 1428 (see, for example, FIG. 14) executed on thesmartphone device to run the computer instructions. In this embodiment,the computer instructions define four variables representing the user'sname (i.e., “John Doe”); the user's vehicle insurance policy number(i.e., “001122334455”); the insurance company's contact number (i.e.,“+1-617-987-6543”); and the emergency help line number (“911”). Theinterpreter module 1428 of smartphone device then obtains the currentlocation information of the user by calling function “GetLocation( )”,returns an output of latitude and longitude from the GPS component ofthe smartphone device, and write the output to variable “Loc” (e.g.,Loc=[+42.12071232, −71.17251217], representing latitude and longitude ofthe accident location). The interpreter module 1428 also obtains thecurrent date and time by calling function “GetCurrentTime( )” andobtains the smartphone device's phone number by calling function“GetPhoneNumber( )”. The outputs of the functions are respectivelystored in variables Time (e.g., “Jan. 1, 2015, 11:59:59 AM”) and Contact(e.g., “+1-617-987-6543”).

In addition, the interpreter module 1428 instructs the smartphone tocall “911” recorded in variable “Police”. After being connected with“911”, the interpreter module 1428 instructs the smartphone to audiblyread out a sentence of, for example, “John Doe had a car accident atlocation [+42.12071232, −71.17251217], on Jan. 1, 2015, 11:59:59 AM andhis/her contact number is +1-617-987-6543, please help.” If the abovesentence is completely read to an officer, an output of SUCCESS willreturn from the function call and is written to the variable Msg1.Otherwise, an output of FAIL will return and write to variable Msg1. Itis appreciated that, instead of calling the police by voice, the abovesentence can be texted to a designated police station by using a “Text()” function. The interpreter module 1428 can additionally oralternatively send a text message to the user's insurance company bycalling a “Text( )” function. In this embodiment, the interpreter module1428 sends to the insurance company a text message string of, forexample, “John Doe, whose policy number is 001122334455, had a caraccident at location [+42.12071232, −71.17251217], on Jan. 1, 2015,11:59:59 AM.” If the above text message string is sent to the insurancecompany, an output of SUCCESS will return from the function call and iswritten to the variable Msg2.

Example Eight—Gaming

The bidirectional display device of the present disclosure can be usedas a touchless user interface. In one scenario, a person playing a videogame may stand or sit in front of a bidirectional display device of thepresent disclosure. For example, referring to FIG. 16A, the gamingsoftware can configure image sensor panel 1610 into two input regions1610L and 1610R. When the player interacts with the video game bygesture, input regions 1610L and 1610R can capture the player's gestureand determine the spatial coordinates of, for example, the player's bodyor hands. Such spatial coordinates can then be used to interact withdifferent game characters or themes in the video game.

For the purposes of describing and defining the present disclosure, itis noted that terms of degree (e.g., “substantially,” “slightly,”“about,” “comparable,” etc.) may be utilized herein to represent theinherent degree of uncertainty that may be attributed to anyquantitative comparison, value, measurement, or other representation.Such terms of degree may also be utilized herein to represent the degreeby which a quantitative representation may vary from a stated reference(e.g., about 10% or less) without resulting in a change in the basicfunction of the subject matter at issue.

Although various embodiments of the present disclosure have beendescribed in detail herein, one of ordinary skill in the art wouldreadily appreciate modifications and other embodiments without departingfrom the spirit and scope of the present disclosure.

What is claimed is:
 1. An image sensor panel, comprising: a transparentsubstrate having a first surface devoid of information display pixels;and a sensor array comprising a plurality of photosensitive pixels onthe first surface and spaced apart from each other; wherein each of thephotosensitive pixels comprises a control component, a photosensitivecomponent, and a dielectric layer between the control component and thephotosensitive component.
 2. The image sensor panel of claim 1, whereinthe sensor array defines a first region on the first surface, withinwhich the photosensitive pixels are disposed, and a second region on thefirst surface other than the first region, and wherein the first regionis optically non-transparent to visible light and the second region isoptically transparent to visible light.
 3. The image sensor panel ofclaim 2, wherein the second region has an area greater than that of thefirst region.
 4. The image sensor panel of claim 1, wherein the controlcomponent is electrically coupled to the photosensitive componentthrough a via hole of the dielectric layer.
 5. The image sensor panel ofclaim 1, wherein the control component comprises at least one thin filmtransistor.
 6. The image sensor panel of claim 1, wherein the controlcomponent, the dielectric layer, and the photosensitive component form avertically stacked structure.
 7. The image sensor panel of claim 6,wherein the vertically stacked structure further comprises a light blocklayer that is optically opaque so as to prevent undesirable light fromreaching the photosensitive component.
 8. An image sensor device,comprising: a planar light source having a light emitting surface; andan image sensor panel disposed directly on the light emitting surface ofthe planar light source, the image sensor panel being devoid ofinformation display pixels and comprising: a transparent substratehaving a first surface, and a sensor array comprising a plurality ofphotosensitive pixels on the first surface and spaced apart from eachother, wherein each of the photosensitive pixels comprises a controlcomponent, a photosensitive component, and a dielectric layer betweenthe control component and the photosensitive component.
 9. The imagesensor of claim 8, wherein the sensor array defines a first region onthe first surface, within which the photosensitive pixels are disposed,and a second region on the first surface other than the first region,and wherein the first region is optically non-transparent to visiblelight and the second region is optically transparent to visible light,thereby allowing at least a portion of light of the planar light sourceto penetrate through the second region.
 10. The image sensor of claim 9,wherein the second region has an area greater than that of the firstregion.
 11. The image sensor of claim 8, wherein the control component,the dielectric layer, and the photosensitive component form a verticallystacked structure.
 12. The image sensor of claim 11, wherein thevertically stacked structure further comprises a light block layer thatis optically opaque so as to prevent light of the planar light sourcefrom reaching the photosensitive pixels.
 13. The image sensor of claim12, wherein the light block layer comprises an electrically conductivematerial electrically coupled with the photosensitive component.
 14. Theimage sensor of claim 8, wherein the control component is electricallycoupled to the photosensitive component through a via hole of thedielectric layer.
 15. The image sensor of claim 8, wherein the controlcomponent comprises at least one thin film transistor.
 16. An electronicdevice, comprising: a display module having one or more substrates, thedisplay module comprising a plurality of display pixels; and an imagesensor panel disposed on the display module, the image sensor panelbeing devoid of information display pixels and comprising: a transparentsubstrate having a first surface, the transparent substrate beingdifferent from said one or more substrates of the display module, and asensor array comprising a plurality of photosensitive pixels on thefirst surface and spaced apart from each other, wherein each of thephotosensitive pixels comprises a control component, a photosensitivecomponent, and a dielectric layer between the control component and thephotosensitive component.
 17. The electronic device of claim 16, whereinthe control component, the dielectric layer, and the photosensitivecomponent form a vertically stacked structure.
 18. The electronic deviceof claim 17, wherein the vertically stacked structure further comprisesa light block layer that is optically opaque so as to prevent lightsignal of the display module from reaching the photosensitive pixels.19. The electronic device of claim 16, wherein the sensor array definesa first region on the first surface, within which the photosensitivepixels are disposed, and a second region on the first surface other thanthe first region, and wherein the first region is optically opaque tovisible light and the second region is optically transparent to visiblelight.
 20. The electronic device of claim 19, wherein the display pixelsof the display module are aligned with the second region of the imagesensor panel.
 21. The electronic device of claim 16, wherein thephotosensitive pixels are configured to have a sensor resolution greaterthan a display resolution of the display module.
 22. The electronicdevice of claim 16, wherein the display module comprises one of (i) anorganic light emitting diode (OLED) display device having a singlesubstrate and (ii) a liquid crystal display (LCD) device having athin-film transistor substrate and a color filter substrate.
 23. Theelectronic device of claim 16, wherein the electronic device is one of asmartphone device, a smart watch device, a handheld video game device, atablet computer, and a laptop computer.